System for bandwidth optimization with initial congestion window determination

ABSTRACT

A system for optimizing network traffic is described. The system includes a transport communication protocol (TCP) controller configured to acquire data regarding a flow of a plurality of data packets over a link and to determine TCP characteristics for the flow, and a congestion window controller configured to determine an initial congestion window based on the TCP characteristics. The TCP controller is further configured to establish a second flow using the initial congestion window.

BACKGROUND

A middlebox is a network appliance that manipulates internet traffic byoptimizing data flow across the network. Middleboxes can be configuredas wide area network (“WAN”) optimizers and can be deployed in pairsacross two geographically separated locations to optimize data trafficbetween the two middleboxes. Middleboxes can be connected through asingle link or multiple links such as a leased line link and a broadbandlink. Middleboxes use TCP congestion avoidance algorithms, commonlycalled “TCP flavors,” to optimize TCP data flows as part of a quality ofservice (“QoS”) scheme. Common examples of TCP avoidance flavors caninclude algorithms such as TCP Vegas, TCP Reno, TCP NewReno, TCP Hybla,TCP BIC, and TCP CUBIC, among others. Each TCP congestion avoidanceflavor is suited for optimizing data flows originating from or receivedby particular operating systems, link types, and/or other networkcharacteristics.

Some TCP flavors improve quality of service across TCP connections by“slow starting” the congestion window size beginning with the initialSYN/ACK cycle, and gradually increasing the congestion window size untilthe network capacity has been reached. Using this scheme, each flowoperating across the connection follows a conservative measure inincreasing the congestion window (cwnd), to fill the network pipe. Forexample, conventional slow start methods begin with a single packet sentand received, then two packets, four packets, etc. Thus, for some TCPflavors, it takes some finite amount of time to reach the networkcapacity, since it can take many round trip times (RTTs) to scale up thecongestion window. In situations where peers are known (for example,across dedicated “leased line” proxy connections between middleboxes) itcan be beneficial to provide systems and methods for optimizingbandwidth using characteristics inherent with these types of congestionavoidance schemes by selecting initial congestion windows according toTCP and link characteristics inherent with a particular connection.

SUMMARY

In some aspects, a system for optimizing network traffic is described.The system includes a transport communication protocol (TCP) controllerconfigured to acquire data regarding a flow of a plurality of datapackets over a link and to determine TCP characteristics for the flow,and a congestion window controller configured to determine an initialcongestion window based on the TCP characteristics. The TCP controlleris further configured to establish a second flow using the initialcongestion window.

In another aspect, a method for optimizing network traffic is described.The method can include acquiring data regarding a flow of a plurality ofdata packets over a link, determining transport communication protocol(TCP) characteristics for the flow, determining an initial congestionwindow based on the TCP characteristics, and establishing a second flowusing the initial congestion window.

In yet another aspect, non-transitory computer readable storage mediumis described. The storage medium stores a set of instructions that areexecutable by at least one processor of an appliance to cause theappliance to perform a method for optimizing network traffic. The methodcan include acquiring data regarding a flow of a plurality of datapackets over a link, determining transport communication protocol (TCP)characteristics for the flow, determining an initial congestion windowbased on the TCP characteristics, and establishing a second flow usingthe congestion window.

BRIEF DESCRIPTION OF THE DRAWINGS

Reference will now be made to the accompanying drawings showing exampleembodiments of this disclosure. In the drawings:

FIG. 1 is a block diagram of an exemplary network environment,consistent with embodiments of the present disclosure.

FIGS. 2A-2B are block diagrams of an exemplary computing device,consistent with embodiments of the present disclosure.

FIG. 3A is a block diagram of an exemplary appliance illustrated in FIG.1, consistent with embodiments of the present disclosure.

FIG. 3B is a block diagram of a portion of an exemplary applianceillustrated in FIG. 3A, consistent with embodiments of the presentdisclosure.

FIG. 4 is a block diagram of an exemplary embodiment for selecting aninitial congestion window, consistent with embodiments of the presentdisclosure.

FIG. 5 is a flowchart representing an exemplary method of modifying aflow, consistent with embodiments of the present disclosure.

FIG. 6 is a flowchart representing an exemplary method of determining aTCP characteristic, consistent with embodiments of the presentdisclosure.

DETAILED DESCRIPTION

Reference will now be made in detail to the exemplary embodimentsimplemented according to the present disclosure, the examples of whichare illustrated in the accompanying drawings. Wherever possible, thesame reference numbers will be used throughout the drawings to refer tothe same or like parts.

The embodiments described herein provide TCP network bandwidthoptimization using initial congestion window determination. The initialcongestion window determination can improve the efficiency of thenetwork data flow through optimization of the bandwidth.

FIG. 1 is a block diagram of an exemplary network environment 100. Whileexemplary network environment 100 is directed to a virtual networkenvironment, it is appreciated that the network environment can be anytype of network that communicates using packets. Network environment 100can include one or more client devices 102, a public network 104, agateway 106, an appliance 108, a private network 110, a data center 120,and a branch office 140.

One or more client devices 102 are devices that can acquire remoteservices from data center 120 through various means. Client devices 102can communicate with a data center 120 either directly (e.g., clientdevice 102 e) or indirectly through a public network 104 (e.g., clientdevices 102 a-d) or a private network 110 (e.g., client device 1020.When client device 102 communicates through public network 104 orprivate network 110, a communication link can be established. Forexample, a link can be established by public network 104, gateway 106,and appliance 108, thereby providing a client device (e.g. clientdevices 102 a-d) access to data center 120. A link can also beestablished by branch office 140 including appliance 108′, privatenetwork 110, and appliance 108, thereby providing a client device (e.g.client device 1020 access to data center 120. While client devices 102are portrayed as a computer (e.g., client devices 102 a, 102 e, and1020, a laptop (e.g., client device 102 b), a tablet (e.g., clientdevice 102 c), and a mobile smart phone (e.g., client device 102 d), itis appreciated that client device 102 could be any type of device (e.g.,wearable or smart watch) that communicates packets to and from datacenter 120.

Public network 104 and private network 110 can be any type of networksuch as a wide area network (WAN), a local area network (LAN), or ametropolitan area network (MAN). As an example, a WAN can be theInternet or the World Wide Web, and a LAN can be a corporate Intranet.Public network 104 and private network 110 can be a wired network or awireless network.

Gateway 106 is a physical device or is software that is part of aphysical device that interfaces between two networks having differentprotocols. Gateway 106, for example, can be a server, a router, a host,or a proxy server. In some embodiments, gateway 106 can include or becoupled to a firewall separating gateway 106 from public network 104(e.g., Internet). Gateway has the ability to modify signals receivedfrom client device 102 into signals that appliance 108 and/or datacenter 120 can understand and vice versa.

Appliance 108 is a device that optimizes wide area network (WAN) trafficby including, for example, a quality of service (“QoS”) engine. In someembodiments, appliance 108 optimizes other types of network traffic,such as local area network (LAN) traffic, metropolitan area network(MAN) traffic, or wireless network traffic. Appliance 108 can optimizenetwork traffic by, for example, scheduling data packets in anestablished communication link so that the data packets can betransmitted or dropped at a scheduled time and rate. In someembodiments, appliance 108 is a physical device, such as Citrix System'sByteMobile™, Netscaler™, or CloudBridge™. In some embodiments, appliance108 can be a virtual appliance. In some embodiments, appliance can be aphysical device having multiple instances of virtual machines (e.g.,virtual Branch Repeater). In some embodiments, a first appliance (e.g.,appliance 108) works in conjunction with or cooperation with a secondappliance (e.g., appliance 108′) to optimize network traffic. Forexample, the first appliance can be located between the WAN and acorporate LAN (e.g., data center 120), while the second appliance can belocated between a branch office (e.g., branch office 140) and a WANconnection. In some embodiments, the functionality of gateway 106 andappliance 108 can be located in a single physical device. Appliances 108and 108′ can be functionally the same or similar. Moreover, in someembodiments, appliance 108 and gateway 106 can be part of the samedevice. Appliance 108 is further described below corresponding to FIG.3A.

Data center 120 is a central repository, either physical or virtual, forthe storage, management, and dissemination of data and informationpertaining to a particular public or private entity. Data center 120 canbe used to house computer systems and associated components, such as oneor more physical servers, virtual servers, and storage systems. Datacenter 120 can include, among other things, one or more servers (e.g.,server 122) and a backend system 130. In some embodiments data center120 can include gateway 106, appliance 108, or a combination of both.

Server 122 is an entity represented by an IP address and can exist as asingle entity or a member of a server farm. Server 122 can be a physicalserver or a virtual server. In some embodiments, server 122 can includea hardware layer, an operating system, and a hypervisor creating ormanaging one or more virtual machines. Server 122 provides one or moreservices to an endpoint. These services include providing one or moreapplications 128 to one or more endpoints (e.g., client devices 102 a-for branch office 140). For example, applications 128 can includeMicrosoft Windows™-based applications and computing resources.

Desktop delivery controller 124 is a device that enables delivery ofservices, such as virtual desktops 126 to client devices (e.g., clientdevices 102 a-f or branch office 140). Desktop delivery controller 124provides functionality required to manage, maintain, and optimize allvirtual desktop communications.

In some embodiments, the services include providing one or more virtualdesktops 126 that can provide one or more applications 128. Virtualdesktops 126 can include hosted shared desktops allowing multiple userto access a single shared Remote Desktop Services desktop, virtualdesktop infrastructure desktops allowing each user to have their ownvirtual machine, streaming disk images, a local virtual machine,individual applications (e.g., one or more applications 128), or acombination thereof.

Backend system 130 is a single or multiple instances of computernetworking hardware, appliances, or servers in a server farm or a bankof servers and interfaces directly or indirectly with server 122. Forexample, backend system 130 can include Microsoft Active Directory™,which can provide a number of network services, including lightweightdirectory access protocol (LDAP) directory services, Kerberos-basedauthentication, domain name system (DNS) based naming and other networkinformation, and synchronization of directory updates amongst severalservers. Backend system 130 can also include, among other things, anOracle™ backend server, a SQL Server backend, and/or a dynamic hostconfiguration protocol (DHCP). Backend system 130 can provide data,services, or a combination of both to data center 120, which can thenprovide that information via varying forms to client devices 102 orbranch office 140.

Branch office 140 is part of a local area network (LAN) that is part ofthe WLAN having data center 120. Branch office 140 can include, amongother things, appliance 108′ and remote backend 142. In someembodiments, appliance 108′ can sit between branch office 140 andprivate network 110. As stated above, appliance 108′ can work withappliance 108. Remote backend 142 can be set up in similar manner asbackend system 130 of data center 120. Client device 102 f can belocated on-site to branch office 140 or can be located remotely frombranch office 140.

Appliances 108 and 108′ and gateway 106 can be deployed as or executedon any type and form of specific computing device (e.g., such as thecomputing device of FIGS. 2A-2B) capable of communicating on any typeand form of network described herein. Appliances 108 and 108′ can bedeployed individually or as a pair operatively connected together.

As shown in FIGS. 2A-2B, each computing device 200 includes a centralprocessing unit (CPU) 221 and a main memory 222. CPU 221 can be anylogic circuitry that responds to and processes instructions fetched fromthe main memory 222. CPU 221 can be a single or multiplemicroprocessors, field-programmable gate arrays (FPGAs), or digitalsignal processors (DSPs) capable of executing particular sets ofinstructions stored in a memory (e.g., main memory 222) or cache (e.g.,cache 240). The memory includes a tangible and/or non-transitorycomputer-readable medium, such as a flexible disk, a hard disk, a CD-ROM(compact disk read-only memory), MO (magneto-optical) drive, a DVD-ROM(digital versatile disk read-only memory), a DVD-RAM (digital versatiledisk random-access memory), flash drive, flash memory, registers,caches, or a semiconductor memory. Main memory 222 can be one or morememory chips capable of storing data and allowing any storage locationto be directly accessed by CPU 221. Main memory 222 can be any type ofrandom access memory (RAM), or any other available memory chip capableof operating as described herein. In the exemplary embodiment shown inFIG. 2A, CPU 221 communicates with main memory 222 via a system bus 250.Computing device 200 can also include a visual display device 224 and aninput/output (I/O) device 230 (e.g., a keyboard, mouse, or pointingdevice) connected through I/O controller 223, both of which communicatevia system bus 250. One of ordinary skill in the art would appreciatethat CPU 221 can also communicate with memory 222 and other devices inmanners other than through system bus 250, such as through serialcommunication manners or point-to-point communication manners.Furthermore, I/O device 230 can also provide storage and/or aninstallation medium for the computing device 200.

FIG. 2B depicts an embodiment of an exemplary computing device 200 inwhich CPU 221 communicates directly with main memory 222 via a memoryport 203. CPU 221 can communicate with a cache 240 via a secondary bus(not shown), sometimes referred to as a backside bus. In some otherembodiments, CPU 221 can communicate with cache 240 via system bus 250.Cache 240 typically has a faster response time than main memory 222. Insome embodiments, such as the embodiment shown in FIG. 2B, CPU 221 cancommunicate directly with I/O device 230 via an I/O port (not shown). Infurther embodiments, I/O device 230 can be a bridge 270 between systembus 250 and an external communication bus, such as a USB bus, an AppleDesktop Bus, an RS-232 serial connection, a SCSI bus, a FireWire™ bus, aFireWire 800™ bus, an Ethernet bus, an AppleTalk™ bus, a GigabitEthernet bus, an Asynchronous Transfer Mode bus, a HIPPI bus, a SuperHIPPI bus, a SerialPlus bus, a SCI/LAMP bus, a FibreChannel™ bus, or aSerial Attached small computer system interface bus, or some other typeof data bus.

As shown in FIG. 2A, computing device 200 can support any suitableinstallation device 216, such as a disk drive or other input port forreceiving one or more computer-readable media such as, for example, aUSB device, flash drive, SD memory card; a hard-drive; or any otherdevice suitable for installing software and programs such as any clientagent 220, or portion thereof. Computing device 200 can further comprisea storage device 228, such as one or more hard disk drives or redundantarrays of independent disks, for storing an operating system and otherrelated software, and for storing application software programs such asany program related to client agent 220. Optionally, any of theinstallation devices 216 could also be used as storage device 228.

Furthermore, computing device 200 can include a network interface 218 tointerface to a LAN, WAN, MAN, or the Internet through a variety of linkincluding, but not limited to, standard telephone lines, LAN or WANlinks (e.g., 802.11, T1, T3, 56 kb, X.25), broadband link (e.g., ISDN,Frame Relay, ATM), wireless connections (Wi-Fi, Bluetooth, Z-Wave,Zigbee), or some combination of any or all of the above. Networkinterface 218 can comprise a built-in network adapter, network interfacecard, PCMCIA network card, card bus network adapter, wireless networkadapter, USB network adapter, modem or any other device suitable forinterfacing computing device 200 to any type of network capable ofcommunication and performing the operations described herein.

FIG. 3A is a block diagram of an exemplary appliance 108 and/or 108′illustrated in FIG. 1, consistent with embodiments of the presentdisclosure. Appliance 108 can include one or more network interfaces218A-N consistent with network interface 218 of FIG. 2A, a QoS engine310, one or more TCP controllers 320, one or more TCP congestion windowcontrollers 322, one or more network traffic detectors 330, a policyengine 346, and a cache manager 350. Although FIG. 2A depicts networkinterfaces 218A-218N as two network interfaces, it is appreciated thatinterfaces 218A-218N can include any number of network interfaces.

QoS engine 310, which is also referred to as a “QoS controller,” or a“QoS packet scheduler,” can perform one or more optimization (e.g.,Quality of Service “QoS”) techniques. QoS engine 310 can be one or moremodules, which can be one or more packaged functional hardware unitsdesigned for use with other components or a part of a program thatperforms a particular function (e.g., optimization techniques),corresponding to the particular step, of related functions. QoS engine310 can be configured to improve the performance, operation, or qualityof service of any type of network traffic. QoS engine 310 performs thesetechniques, for example, by using defined logic, business rules,functions, or operations. In some embodiments, QoS engine 310 canperform network traffic optimization and management mechanisms thatprovide different priorities to different users, applications, flows, orlinks. QoS engine 310 can also control, maintain, or assure a certainlevel of performance to a user, application, flow, or connection. QoSengine 310 can direct TCP controller 320 to perform any or all steps fordetermining one or more initial congestion windows using one or more TCPcharacteristics. For example, QoS engine 310 can control, maintain, orassure a certain portion of bandwidth or network capacity of acommunication link for a user, application, one or more flows, or links,and/or collect data in connection with one or more flows and links,analyze the collected data.

In some embodiments, QoS engine 310 can monitor the achieved level ofperformance or the quality of service (e.g., the data rate and delay)corresponding to a user, application, and/or flow, or link, and thendynamically control or adjust one or more TCP characteristics inconnection with sending and receiving data packets to achieve thedesired level of performance or quality of service. QoS engine 310 candirect TCP controller 320 to perform some or all of the steps accordingto exemplary embodiments disclosed herein. For example, QoS engine 310can coordinate the acquisition and delivery of TCP characteristicsbetween TCP congestion window controller 322 and TCP controller 320. QoSengine 310 can also coordinate the acquisition and delivery of linkcharacteristics between components of appliance 108, such as, forexample, between network traffic detector 330, TCP controller 320, andTCP congestion window controller 322.

TCP controller 320, which is also referred to as a “packet engine,” a“packet processor,” or a “data processor,” is responsible forcontrolling and managing the processing of data packets received andtransmitted by appliance 108 via network interfaces 218A-N. TCPcontroller 320 can be one or more modules, which can be one or morepackaged functional hardware units designed for use with othercomponents or a part of a program that performs a particular function(e.g., controlling and managing the processing of data packets),corresponding to the particular step, of related functions. TCPcontroller 320 can be embodied as a single packet engine or any numberof a plurality of packet engines that can operate at the data link layer(layer 2), network layer (layer 3), or the transport layer (layer 4) ofa network stack (e.g., such as the layers and protocols of the OpenSystem Interconnection communications model). TCP controller 320 can beconfigured to accomplish some or all of the steps described herein afterbeing executed by CPU 221 and/or QoS engine 310. In some aspects, thedata packets can be carried over the data link layer via the Ethernetcommunication protocol, which can comprise any of the family of WAN orLAN protocols, such as those protocols covered by the IEEE 802.3. Inother aspects, the network stack can have any type and form of wirelessprotocols, such as IEEE 802.11 and/or mobile internet protocols. In someembodiments, TCP controller 320 intercepts or receives data packets atthe network layer, such as via the IP communication protocol. In someembodiments, TCP controller 320 can intercept or receive data packets atthe transport layer, such as via the TCP communication protocols. TCPcontroller 320 can operate at any session or any application layer abovethe transport layer.

TCP controller 320 can include a buffer for queuing one or more datapackets during processing of the data packets. Additionally, TCPcontroller 320 can communicate via one or more communication protocolsto transmit and receive a plurality of network data packets across oneor more links via network interfaces 218A-N. The links can connectappliance 108 to 108′. TCP controller 320 can be configured to acquiredata regarding the flow and store, the acquired data in an operativelyconnected computer memory. The sent and received data packets operatingacross one or more links can be considered “data flows” or “flows.” Insome embodiments, TCP controller 320 can send scheduling requests to QoSengine 310 for scheduling of data packets received and stored at TCPcontroller 320. After TCP controller 320 receives responses from QoSengine 310, TCP controller 320 processes the stored data packetsaccording to their scheduled priorities. TCP controller 320 candetermine one or more TCP characteristics of the flow based on thestored data. A TCP characteristic, as discussed in further detail below,includes a plurality of information such as, for example, packet roundtrip times and/or the packet loss rate for a particular data flow, anaverage queuing delay and/or bandwidth delay product for the packetssent and received across a particular link, congestion window dropinformation, and/or other congestion window information such ascongestion window size, among other things.

During operations of appliance 108, TCP controller 320 can interface, beintegrated with, or be in communication with any portion of appliance108, such as QoS engine 310, TCP congestion window controller 322,network traffic detector 330, policy engine 346, and/or cache manager350. As such, any of the logic, functions, or operations of QoS engine310, TCP congestion window controller 322, network traffic detector 330,policy engine 346, and/or cache manager 350 can be performed inconjunction with or in responsive to TCP controller 320. CPU 221 cancontrol by and/or execute any operation described herein.

In some aspects, one or more TCP congestion window controllers 322 canbe configured to send and receive flow information from TCP controller320, and/or QoS engine 310. TCP congestion window controller 322 can beconfigured to acquire one or more TCP characteristics from TCPcontroller 320 and select a TCP flavor based on the TCP characteristics.Because the flow characteristics change with time during the initialcongestion window determination process, the selection is said to be“dynamic.” TCP characteristics can include one or more characteristicsthat change with time, such as, for example, packet round trip timesand/or the packet loss rate for a particular data flow, an averagequeuing delay for the packets sent and received across a particularlink, and/or congestion window information. TCP congestion windowcontroller 322 can be one or more modules, which can be one or morepackaged functional hardware units designed for use with othercomponents or a part of a program that performs a particular function(e.g., controlling and managing the processing of data packets),corresponding to the particular step, of related functions.

One or more network traffic detectors 330 can include any logic,business rules, functions, or operations for automatically detecting thetype of network traffic corresponding to data packets acquired by TCPcontroller 320. Network traffic detector 330 can be one or more modules,which can be one or more packaged functional hardware units designed foruse with other components or a part of a program that performs aparticular function (e.g., acquire one or more link characteristics),corresponding to the particular step, of related functions. As describedabove, TCP controller 320 can store and transmit data packets from anytype of network traffic, such as data packets from any communicationprotocols including WAN, MAN, LAN, and wireless communication protocols.In some embodiments, not all network traffic is optimized by QoS engine310. For example, QoS engine 310 can be used to optimize the WANtraffic, but not the LAN traffic or traffic directed to management.Network traffic detector 330 can detect the type of network trafficreceived at TCP controller 320 by any available techniques, such as byusing IP addresses. Network traffic detectors 330 can also determine alink type, a bandwidth, and/or other characteristics associated with oneor more flows.

Appliance 108 can also include a policy engine 346, also referred to asa policy controller or a policy provider. Policy engine 346 can includeany logic, function, or operations for providing and applying one ormore policies or rules to the function, operation, or configuration ofany portion of the appliance 108. Policy engine 346 can be one or moremodules, which can be one or more packaged functional hardware unitsdesigned for use with other components or a part of a program thatperforms a particular function, corresponding to the particular step, ofrelated functions. In some embodiments, policy engine 346 provides aconfiguration mechanism to allow a user to identify, specify, define, orconfigure a policy for appliance 108, or any portion thereof. Forexample, policy engine 346 can provide a predefined traffic optimizationconfiguration policy including the number of priorities, the prioritiesassociated with each service class, the number of connections allowedunder each service class, link bandwidth configuration, and any otherpolicy information. Policy engine 346 can also provide policies for whatdata to cache, when to cache the data, for whom to cache the data, whento expire an object in cache, or when to refresh the cache. Policyengine 346 can also include any logic, rules, functions, or operationsfor determining and providing access, control, and management of datapackets received and stored by TCP controller 320. Policy engine 346 canalso include any logic, rules, functions, or operations for determiningand providing access, control and management of security, networktraffic, network access, compression, or any other function or operationperformed by appliance 108.

Cache manager 350 can include software, hardware, or any combination ofsoftware and hardware to store data, information, and objects to a cachein memory or storage; to provide cache access; and to control and managethe cache. The data, objects, or content processed and stored by cachemanager 350 can include data in any format, such as a six-byte MACaddress, a TCP data packet, or any type of data communicated via anycommunication protocol. Examples of types of data can include, forexample, one or more TCP characteristics including information inconnection with packet loss rates, queuing delays, flow congestion,sizes of congestion windows, bandwidth of one or more links, averageround trip times, etc. Cache manager 350 can duplicate original datastored in a slow-access storage and store the data in a fast-accesscache memory, such as cache 240. After the data is stored in the cache,future use can be made by accessing the cached copy rather thanrefetching or recomputing the original data, thereby reducing the accesstime. In some embodiments, the cache can comprise a data object inmemory of appliance 108. In some embodiments, the cache can comprise anytype and form of storage element of appliance 108, such as a portion ofa hard disk. In some embodiments, as described above, the processingunit of the device, such as CPU 221, can provide cache memory for use bycache manager 350. Cache manager 350 can use any portion and combinationof main memory 222, storage 228, or CPU 221 for caching data, objects,and other content. Cache manager 350 can comprise any type of generalpurpose processor (GPP), or any other type of integrated circuit, suchas a Field Programmable Gate Array (FPGA), Programmable Logic Device(PLD), or Application Specific Integrated Circuit (ASIC). Cache manager350 can be one or more modules, which can be one or more packagedfunctional hardware units designed for use with other components or apart of a program that performs a particular function, corresponding tothe particular step, of related functions.

FIG. 3B is a block diagram of a portion of exemplary appliance 108illustrated in FIG. 3A, consistent with embodiments of the presentdisclosure. In some embodiments, the operating system of appliance 108allocates, manages, or otherwise segregates the available system memoryinto what is referred to as kernel space (system space) and user space(application space). The kernel space is typically reserved for runningthe kernel, including any device drivers, kernel extensions, or otherkernel related software. The kernel can be the core of the operatingsystem, and provides access, control, and management of resources andhardware-related elements of the appliance 108. In some aspects, thekernel space can also include a number of network services or processesworking in conjunction with QoS engine 310, TCP controller 320, TCPcongestion window controller 322, or any portion thereof. Additionally,the embodiments of the kernel can depend on the operating systeminstalled, configured, or otherwise used by appliance 108.

User space is the memory area or portion of the operating system used byuser mode applications or programs otherwise running in user mode. Auser mode application cannot access kernel space directly and usesservice calls to access kernel services. The operating system uses theuser space for executing or running applications and provisioning ofuser level programs, services, processes, and/or tasks. As an example,the operating system can execute software of network interfaces 218A-Nin the user space.

FIG. 4 is a block diagram of an exemplary embodiment for selecting aninitial TCP congestion window 440, consistent with embodiments of thepresent disclosure. TCP congestion window controller 322 can beconfigured to receive both static input and dynamic input, and use bothinputs to determine an initial TCP congestion window 440. Static inputcan include one or more TCP link characteristics that includeinformation regarding one or more links across which one or more activeflow 450 are operating. Examples of a TCP link characteristic caninclude bandwidth information (e.g., bandwidth 413), link type (e.g.,412), and/or the number of active TCP connections, among other things.TCP characteristics 430 can also include dynamically-changinginformation in connection with packet loss rates, queuing delays, flowcongestion, sizes of congestion windows, average round trip times,and/or other information in connection with active flow 450. A flow is“active” when packets are being sent and received across a TCP link.

In TCP connections, the congestion window is one of the factors used todetermine the number of bytes that can be outstanding in an active flowat a given time. Congestion windows are also a means of stopping a linkbetween two link terminals (e.g., between appliance 108 and 108′) frombeing overloaded with too much traffic. The congestion window size canbe determined by estimating how much TCP packet congestion there isbetween the two link terminals. The data sender generally maintains anddetermines the congestion window size. When a new TCP connection isestablished, the value for the congestion window size is maintainedindependently at each host. The appliance establishing the TCPconnection can set the congestion window size to a small multiple of themaximum segment size (MSS) allowed on that connection. Further variancein the congestion window is dictated by an “additiveincrease/multiplicative decrease” approach. This means that if allsegments are received and the acknowledgments reach the sender on time,a multiple of some constant is added to the window size. The windowkeeps growing exponentially until a timeout occurs or the receiverreaches its limit (a threshold value “ssthresh”).

One example of exponential congestion window growth is the “slow-start”algorithm used by TCP to control congestion inside the network.Conventional slow-start algorithms generally begin with an initialcongestion window (cwnd) size of 1, 2, or 4 packets. The value of thecongestion window will be increased with each acknowledgement received,which doubles the window size with each round trip time. Thetransmission rate is generally increased until either a packet loss isdetected, or a receiver's advertised window is reached, or the slowstart threshold is reached. If packet loss occurs, conventionalcongestion avoidance algorithms can assume that it is due to networkcongestion. The system (e.g., the apparatus establishing the TCPconnection) can then take steps to reduce the offered load on thenetwork. Once the slow-start threshold is reached, conventional TCPalgorithms change from the “slow-start” algorithm to the “linear growth”(congestion avoidance) algorithm (i.e., a TCP flavor). During the lineargrowth phase, the congestion window is conventionally increased by aconstant number (usually 1 packet) for each round trip time.

Referring again to FIG. 4, an exemplary TCP congestion window controller322 can be configured to control the initial congestion window size ofnascent proxied TCP flows, and control the initial congestion windowsize of the new flows based on dynamically-changing TCP characteristicsderived empirically from a sampled list of previously seen long-livedTCP links by recording the congestion related parameters for eachsampled flow. Although presently disclosed embodiments refer to a“nascent” link and/or a “second” link, it is appreciated thatembodiments disclosed herein can be configured to establish multiple newTCP flows that may exceed several thousand in number. In some aspects,appliance 108 can improve network speed, efficiency and quality ofservice by selecting an initial congestion window for nascent TCP flowsthat avoids many of the round trip times associated with conventionalslow start algorithms.

In some aspects, TCP controller 320 is configured to send and receive aplurality of data packets via a flow operating across a proxied link(“leased line”) link, and store flow information indicative of variousoperational aspects of the flow in an operatively connected computermemory (e.g., main memory 222). TCP controller 320 can determine one ormore link characteristics using link information (e.g., link type 412and/or bandwidth 413), and determine one or more TCP characteristics forthe flow using the flow information and link characteristics.

According to some embodiments, one or more processors (e.g., CPU 221)can execute TCP congestion window controller 322. TCP congestion windowcontroller 322 can then receive TCP characteristics 430 (which aredynamically changing over time), receive static input (which does notchange over time), and select an initial TCP congestion window 440 basedon the static input and/or dynamic input. For example, TCP congestionwindow controller 322 can acquire one or more TCP characteristics 430from TCP controller 320, acquire one or more link characteristics (e.g.,link type 412 and/or bandwidth 413) from network traffic detector 330,and select initial TCP congestion window 440.

In some aspects, TCP controller 320 can continually monitor the trafficfor a predetermined period of time, and continually provide dynamicfeedback to TCP congestion window controller 322. Although apredetermined period of time can vary based on application, it iscontemplated that TCP controller 320 can monitor traffic for periods ofseveral seconds to periods of time spanning several minutes beforecalculating TCP characteristics 430.

Using link characteristics and TCP characteristics 430, appliance 108can estimate the amount of data that could be pushed to the network whenthe connection starts without causing congestion, and better gain ofthroughput for each flow while being TCP fair to other flows operatingacross the active links. By changing the initial TCP congestion window440 for the nascent TCP links, some embodiments can reduce the totalnumber of round trip times (RTTs) needed to bring the new TCP links upto full bandwidth capacity. Accordingly, appliance 108 modifies flow 450by determining an initial TCP congestion window 440 for each new link.In some aspects, appliance 108 provides greater TCP throughput, fewerlost packets, and an improved user experience due to system speed andstability.

FIG. 5 is a flowchart representing an exemplary method 500 for modifyinga flow, consistent with embodiments of the present disclosure. It willbe readily appreciated that the illustrated procedure can be altered todelete steps or further include additional steps. While method 500 isdescribed as being performed by appliance (e.g., appliance 108), it isappreciated that method 500 can be performed by other devices alone orin combination with the appliance. After an initial start step 510,appliance 108 sends and receives a plurality of data packets comprisingan active flow that are operating across a link. At step 520, appliance108 can acquire and store information regarding the active flow to anoperatively connected computer-readable memory. According to someembodiments, appliance 108 can also determine other informationindicative of the link across which the active flows are operating (step530). A link type can be, for example, a broadband link, a dedicatedleased-line link between two dedicated apparatuses (e.g., 180 and 180′),and/or other types of links across which active flow 450 operates. Linkcharacteristics can include information indicative of the link (e.g.,link type 412) and/or bandwidth information (e.g., bandwidth 413). It iscontemplated that appliance 108 can acquire link characteristics usingnetwork traffic detector 330, or any other operatively configured moduleon the appliance.

After determining a link characteristic, appliance 108 can determine TCPcharacteristics (step 540) based on the stored flow data. FIG. 6considers an exemplary method 600 for determining TCP characteristics(e.g., TCP characteristics 430).

Referring now to FIG. 6, an exemplary method 600 for determining TCPcharacteristics is described, consistent with embodiments of the presentdisclosure. After an initial starting step 600, appliance 108 candetermine information about the link type, determine information aboutthe flow, and determine aspects of the TCP traffic and operation of theTCP link. For example, at step 610, appliance 108 can determine, basedon the link characteristics, whether the active link is a “leased”(proxied) type connection. If appliance 108 determines that the linktype is not a leased connection, method 600 terminates at step 650. Ifthe link type is a leased line type link, method 600 can proceed to step620.

At step 620, appliance 108 can determine a low bandwidth delay product(BDP_(low)) for the shortest round trip time (RTT_(low)) of active flow450. The BDP_(low) determination considers bandwidth (B) of a link, andaverage round trip time (RTT_(low)) for the plurality of packetsrecently sent and received by TCP controller 320 across a particularlink. BDP_(low) is determined byBDP_(low) =B*RTT_(low),where B is a bandwidth of the link (e.g., bandwidth 413), and RTT_(low)is the minimum round trip time for the plurality of data packetsoperating across the link during the predetermined time interval. Whilethe determination of the low bandwidth delay product can involve acalculation (as shown above), it is appreciated that the determinationcan involve other mechanisms, such as using a look-up table based oninputs B and RTT_(low).

At step 630, appliance 108 can determine a total congestion window(“cwnd_(total)”) for each active flow. A total congestion window givesthe sum of all current congestion window sizes for each of the activeflows. At step 630, appliance 108 can cause TCP controller 320 todetermine a total congestion window (“cwnd_(total)”), wherecwnd_(total)=cwnd_(current) _(_) ₁+cwnd_(current) _(_) ₂+ . . .cwnd_(current) _(_) _(n).Each respective value “cwnd_(current) _(_) _(n)” is an active congestionwindow of the flow operating across the TCP link. The total congestionwindow provides an approximate value for the total number of packetsoccupying an active link at any given time. While the determination ofthe total congestion window can involve a calculation (as shown above),it is appreciated that the determination can involve other mechanisms,such as using a look-up table.

In some aspects, at step 640 appliance 108 can determine a congestionwindow drop average (cwnd_(avgdrop)). The congestion window drop(cwnd_(drop)) provides information regarding the current congestionwindow after a packet drop has occurred. The value cwnd_(avgdrop) givesthe average of all of the dropped congestion windows over apredetermined interval of time for an active flow, wherecwnd_(avgdrop)=average of cwnd_(drop).In some embodiments, each flow operating across an active link tends toconverge to a point of equilibrium in order to achieve maximumthroughput between the congestion window where a packet drop hasoccurred, and the reduced congestion window after the packet drop. Thusa variable for the average dropped congestion window (cwnd_(avgdrop))can identify the average of the lower bound for the flows to convergefor maximum throughput.

Referring again to FIG. 5, after appliance 108 determines TCPcharacteristics 430, appliance 108 can acquire the link characteristicsfrom network traffic detector 330 (step 550) and TCP characteristics 430from TCP controller 320 (step 560). Although TCP congestion windowcontroller 322 is said to acquire the flow characteristic, it iscontemplated that TCP controller 320 can also provide or transmit TCPcharacteristics 430 in response to a request, a call, etc., from TCPcongestion window controller 322. For example and as previouslydiscussed, TCP characteristics 430 can dynamically change with respectto time. In some aspects, TCP controller 320 can acquire the flow datasaved over a predetermined time interval (e.g., 10 seconds, 10 minute,30 minutes, etc.), and continually determine TCP characteristics 430 ateach predetermined interval. Additionally and/or alternatively,appliance 108 can provide/transmit TCP characteristics 430 to TCPcongestion window controller 322 at predetermined time intervals.

According to some embodiments, TCP controller 320 determines TCPcharacteristics 430 using variables BDP_(low), cwnd_(avgdrop), andcwnd_(total), which are based on the flow information stored for thepredetermined time interval. Using these variables, TCP congestionwindow controller 322 can determine an initial congestion window (step570) for a new TCP connection by comparing BDP_(low) with cwnd_(total)and cwnd_(avgdrop). TCP congestion window controller 322 can determinewhether BDP_(low) is greater than cwnd_(total). If BDP_(low) issignificantly greater than cwnd_(total)+cwnd_(avgdrop), TCP congestionwindow controller 322 can determine that the active link is notcongested, and set an initial congestion window (IW) accordingly. Forexample, when the active link is not congested, the IW for the new flowcould be cwnd_(avgdrop). Since cwnd_(avgdrop) provides the average ofthe smallest cwnd for the active flows to converge for maximumthroughput, TCP congestion window controller 322 can set the value ofcwnd_(avgdrop) as the initial congestion window for the new flow.

In some embodiments, TCP appliance 108 can determine that there isenough capacity in the bandwidth delay product (BDP), and the initialcongestion window set would not cause packet drops due to congestion.Accordingly, the initial congestion window set would be “TCP fair” tothe new TCP connections. Method 500 can constantly adapt the initialcongestion window for the new connections depending on the overallbandwidth delay product. When the bandwidth delay product is approachingfull capacity, appliance 108 can revert back to the standard TCPalgorithm and establish new TCP connections using a “standard” initialcongestion window as determined by the TCP congestion avoidancealgorithm (e.g., a TCP congestion avoidance flavor).

According to some embodiments, at step 570, TCP appliance 108 candetermine an initial congestion window using dynamically-changing linkcharacteristic fed back from TCP controller 320. The followingabstraction is an exemplary algorithm for determining an initialcongestion window (IW) based on dynamic feedback from TCP controller320:

-   -   if ((cwnd_(total)+cwnd_(avgdrop))<50% of BDP_(low))/*Path is not        congested*/        -   IW=cwnd_(avgdrop);    -   else if ((cwnd_(total)+cwnd_(avgdrop))<75% of BDP_(low))/*Path        can get congested*.        -   IW=α*cwnd_(avgdrop); /*where 0<α<1*/    -   else        -   IW=4.            In the above exemplary algorithm, “IW” is an initial            congestion window, and α is a value between 0 and 1. Based            on the above exemplary algorithm, TCP congestion window            controller 322 can determine the IW size for the next (new)            TCP connection based on a dynamically changing link            characteristic provided by TCP controller 320.

In the foregoing specification, embodiments have been described withreference to numerous specific details that can vary from implementationto implementation. Certain adaptations and modifications of thedescribed embodiments can be made. Other embodiments can be apparent tothose skilled in the art from consideration of the specification andpractice of the embodiments disclosed herein. It is intended that thespecification and examples be considered as exemplary only. It is alsointended that the sequence of steps shown in figures are only forillustrative purposes and are not intended to be limited to anyparticular sequence of steps. As such, those skilled in the art canappreciate that these steps can be performed in a different order whileimplementing the same method.

What is claimed is:
 1. A first network appliance for optimizing networktraffic over a link between the first network appliance and a secondnetwork appliance, comprising: a transport communication protocol (TCP)controller configured to: acquire data regarding a plurality of activedata-packet flows over the link, wherein the link is a proxied linkbetween the first network appliance and the second network appliance andwherein each active data-packet flow of the plurality of active datapacket flows corresponds to a corresponding congestion window, determineTCP characteristics for each flow of the plurality of active data-packetflows, determine a congestion window size for each of the congestionwindows that have dropped a data packet from a plurality of data packetsduring a predetermined period of time, and determine a congestion windowdrop average based on an average of the congestion window sizes for eachof the congestion windows that have dropped data packets from theplurality of data packets during the predetermined period of time; and acongestion window controller configured to determine an initialcongestion window for a new flow of data packets over the link based onthe TCP characteristics, a bandwidth of the link, a total congestionwindow size corresponding to the plurality of active data-packet flows,and the congestion window drop average, wherein the TCP controller isfurther configured to establish the new flow using the initialcongestion window.
 2. The first network appliance of claim 1, whereinthe TCP controller acquires the data at a predetermined interval oftime.
 3. The first network appliance of claim 1, wherein the TCPcontroller is further configured to: determine, using the data regardingthe plurality of active data-packet flows: a bandwidth delay value basedon the bandwidth of the link and the TCP characteristics for each flowof the plurality of active data-packet flows; the total congestionwindow size; and the congestion window drop average.
 4. The firstnetwork appliance of claim 3, wherein the TCP controller is furtherconfigured to: determine a size of the congestion window for each activedata-packet flow of a plurality of active data-packet flows on the link;determine the total congestion window size based on the sizes of thecongestion windows for the plurality of active data-packet flows.
 5. Thefirst network appliance of claim 3, wherein the TCP controller isfurther configured to: determine a smallest duration of time for aplurality of round trips for data packets sent and receivedcorresponding to each flow of the plurality of active data-packet flow;and determine a low bandwidth delay value using a product of thesmallest duration of time for the plurality of round trips and thebandwidth of the link.
 6. The first network appliance of claim 5, theTCP controller is further configured to: evaluate a sum of the totalcongestion window size and the congestion window drop average; comparethe sum to the low bandwidth delay product; and determine the initialcongestion window based on the comparison.
 7. A method for optimizingnetwork traffic over a link between a first network appliance and asecond network appliance, wherein the method is performed by the firstnetwork appliance and comprising: acquiring data regarding a pluralityof active data-packet flows over the link, wherein the link is a proxiedlink between the first network appliance and the second networkappliance and wherein each active data-packet flow of the plurality ofactive data packet flows corresponds to a corresponding congestionwindow; determining transport communication protocol (TCP)characteristics for each flow of the plurality of active data-packetflows; determining a congestion window size for each of the congestionwindows that have dropped a data packet from a plurality of data packetsduring a predetermined period of time, and determining a congestingcongestion window drop average based on an average of the congestionwindow sizes for each of the congestion windows that have dropped datapackets from the plurality of data packets during the predeterminedperiod of time; determining an initial congestion window for a new flowof data packets over the link based on the TCP characteristics, abandwidth of the link, a total congestion window size corresponding tothe plurality of active data-packet flows, and the congestion windowdrop average; and establishing the new flow using the initial congestionwindow.
 8. The method of claim 7, further comprising acquiring the dataat a predetermined interval of time.
 9. The method of claim 7, furthercomprising: determining, using the data regarding the plurality ofactive data-packet flows: a bandwidth delay value based on the bandwidthof the link and the TCP characteristics for each flow of the pluralityof active data-packet flows; the total congestion window size; and thecongestion window drop average.
 10. The method of claim 9, furthercomprising: determining a size of the congestion window for each activedata-packet flow of a plurality of active data-packet flows on the link;determining the total congestion window size based on the sizes of thecongestion windows for the plurality of active data-packet flows. 11.The method of claim 9, further comprising: determining a smallestduration of time for a plurality of round trips for data packets sentand received corresponding to each flow of the plurality of activedata-packet flow; and determining a low bandwidth delay value using aproduct of the smallest duration of time for the plurality of roundtrips and the bandwidth of the link.
 12. The method of claim 11, furthercomprising: evaluating a sum of the total congestion window size and thecongestion window drop average; comparing the sum to the low bandwidthdelay product; and determining the initial congestion window based onthe comparison.
 13. A non-transitory computer readable storage mediumthat stores a set of instructions that are executable by at least oneprocessor of a first network appliance to cause the first networkappliance to perform a method for optimizing network traffic over a linkbetween the first network appliance and a second network appliance, themethod comprising: acquiring data regarding a plurality of activedata-packet flows over the link, wherein the link is a proxied linkbetween the first network appliance and the second network appliance andwherein each active data-packet flow of the plurality of active datapacket flows corresponds to a corresponding congestion window;determining transport communication protocol (TCP) characteristics foreach flow of the plurality of active data-packet flows; determining acongestion window size for each of the congestion windows that havedropped a data packet from a plurality of data packets during apredetermined period of time, and determining a congestion window dropaverage based on an average of the congestion window sizes for each ofthe congestion windows that have dropped data packets from the pluralityof data packets during the predetermined period of time; determining aninitial congestion window for a new flow of data packets over the linkbased on the TCP characteristics, a bandwidth of the link, a totalcongestion window size corresponding to the plurality of activedata-packet flows, and the congestion window drop average; andestablishing the new flow using the initial congestion window.
 14. Thenon-transitory computer readable storage medium of claim 13, wherein theset of instructions that are executable by the at least one processor ofthe first network appliance to cause the first network appliance tofurther perform: acquiring the data at a predetermined interval of time.15. The non-transitory computer readable storage medium of claim 13,wherein the set of instructions that are executable by the at least oneprocessor of the first network appliance to cause the first networkappliance to further perform: determining, using the data regarding theplurality of active data-packet flows: a bandwidth delay value based onthe bandwidth of the link and the TCP characteristics for each flow ofthe plurality of active data-packet flows; the total congestion windowsize; and the congestion window drop average.
 16. The non-transitorycomputer readable storage medium of claim 15, wherein the set ofinstructions that are executable by the at least one processor of thefirst network appliance to cause the first network appliance to furtherperform: determining a size of the congestion window for each activedata-packet flow of a plurality of active data-packet flows on the link;determining the total congestion window size based on the sizes of thecongestion windows for the plurality of active data-packet flows. 17.The non-transitory computer readable storage medium of claim 15, whereinthe set of instructions that are executable by the at least oneprocessor of the first network appliance to cause the first networkappliance to further perform: determining a smallest duration of timefor a plurality of round trips for data packets sent and receivedcorresponding to each flow of the plurality of active data-packet flow;and determining a low bandwidth delay value using a product of thesmallest duration of time for the plurality of round trips and thebandwidth of the link.
 18. The non-transitory computer readable storagemedium of claim 17, wherein the set of instructions that are executableby the at least one processor of the first network appliance to causethe first network appliance to further perform: evaluating a sum of thetotal congestion window size and the congestion window drop average;comparing the sum to the low bandwidth delay product; and determiningthe initial congestion window based on the comparison.